Issues for Corpus-Based Multimodal Generation
نویسنده
چکیده
In recent years, data-driven methods have become increasingly popular in natural language generation. Multimodal generation can also benefit from using corpus data directly; however, there are several issues that arise when using corpora for multimodal generation that do not occur in the unimodal case, and that mean that existing multimodal corpora are often not suitable for being directly used in a generation system.
منابع مشابه
Creating an Annotated Corpus for Generating Walking Directions
This work describes first steps towards building a system that synchronously generates multimodal (textual and visual) route directions for pedestrians. We pursue a corpus-based approach for building a generation model that produces natural instructions in multiple languages. We conducted an empirical study to collect verbal route directions, and annotated the acquired texts on different levels...
متن کاملUnnatural Language Processing an Empirical Study of Multimodal Proof Styles
Computer-based logic proofs are a form ofùnnatural' language in which the process and structure of proof generation can be observed in considerable detail. We have been studying how students respond to multimodal logic teaching, and performance measures have already indicated that students' pre-existing cognitive styles have a signiicant impact on teaching outcome. Furthermore, a large corpus o...
متن کاملMultimodal Corpus Collection for the Design of User-Programmable Robots
In order to design a robot able to learn from its user, the example of card game instruction is investigated. A detailed description of a setup for the collection of a human-to-human multimodal instruction corpus is given. This setup uses touch screens and will also provide a base for the human-robot instruction interface to be designed. Preliminary results on learning dialogues are given and i...
متن کاملTowards Generation Of Fluent Referring Action In Multimodal Situations
Referring actions in multimodal situations can be thought of as linguistic expressions well coordinated with several physical actions. In this paper, what patterns of linguistic expressions are commonly used and how physical actions are temporally coordinated to them are reported based on corpus examinations. In particular, by categorizing objects according to two features, visibility and membe...
متن کاملThe SmartKom Multimodal Corpus at BAS
In this contribution we announce and describe in detail the new multimodal corpus evolving from the publicly funded German SmartKom project. The first release of the corpus (BAS SK-P 1.0) has been finished end of 2001 and will be ready for distribution to the scientific community in July 2002. The SmartKom corpus will be the first of a new generation of Language Resources (LR) designed for a mo...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2006